The Infiniband® network includes nodes that communicate through a channel-based switched fabric. (Infiniband® is a registered trademark of Infiniband Trade Association, located in Beaverton, Oreg.). For example, the nodes may be a host, an input/output subsystem, or a router which connects to another network. The switched fabric is made of a collection of switches, routers, and/or links that connect a set of channel adapters. The channel adapters form an interface between the switched fabric and the nodes. The channel adapter of the host is referred to as a host channel adapter (HCA). The channel adapter of an input/output (I/O) subsystem is referred to as a target channel adapter.
In Infiniband®, two processes communicate using a queue pair. A queue pair (QP) includes a send queue and a receive queue. Specifically, in order for a process to send a message to another process, the process posts the message to the send queue. More specifically, per the Infiniband® protocol, each message transmitted/received has a corresponding send queue and a corresponding receive queue. An application uses the send queue to send messages and the receive queue to receive messages. The send queue and receive queue on the same host that is used by the application to communicate with another application form a QP. Each QP may have a corresponding QP with which to communicate. For example, consider the scenario where application M is communicating with application N. In such a scenario, application M may have QP M, with send queue M and receive queue M, and application N may have QP N, with send queue N and receive queue N. Messages from application M to application N are sent from send queue M to receive queue N. Messages from application N to application M are sent from send queue N to receive queue M. Logic and data structures used by the host system specify which QP on the recipient to use to send messages. Thus, by the requestor application specifying the QP, the requestor application is identifying the responder application to receive the message. Each packet that is sent may include a packet sequence number. Logic associated with the receive queue ensures that packets are processed in a particular order using the packet sequence number.
Infiniband® supports virtual-to-physical address translation services for translations associated with host memory accesses. When an I/O device, such as the HCA, requests to access host memory, the I/O device typically obtains a translation for one or more HCA structures (e.g., tables or other data structures to which the HCA may be writing or reading from) from hardware located in the host. The I/O device may store/cache the obtained translation in the I/O device itself for future use.